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The theory of Extreme Physical Information (EPI) is used to deduce a probability density function 
(PDF) of a system that exhibits a power law tail. The computed PDF is useful to study and fit 
several observed distributions in complex systems. With this new approach it is possible to describe 
extreme and rare events in the tail, and also the frequent events in the distribution head. Using 
EPI, an information functional is constructed, and minimized using Euler-Lagrange equations. As 
a solution, a second order differential equation is derived. By solving this equation a family of 
functions is calculated. Using these functions it is possible to describe the system in terms of 
eigenstates. A dissipative term is introduced into the model, as a relevant term for the study of 
open systems. One of the main results is a mathematical relation between the scaling parameter of 
the power law observed in the tail and the shape of the head. 
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I. INTRODUCTION 

In the natural and social sciences there are many variables that distribute as power laws [1, 2]. Several models 
^-», are described in the literature to explain and fit these power laws. There are models from first principles [3-7], 
' microscopic interactions [8-10], using stochastic models [11, 12] and even ad hoc mathematical formulas [13]. The 
problem identified in many of these models is the lack of analysis of frequent events in the distribution head and the 
possible relations between head events and extreme events in the tail. The aim of this paper is to construct a model 
£NJ \ that simultaneously describes the power law in the tail and the shape of the head. 

Reed [11] has proposed a model that describes and fits the head and tail of a social variable. Reed's model is 
stochastic with independent parameters for head and tail. This article describes a model based on basic information 
principles, from which a relation between the head and the tail parameters is derived. 

The model builds on the basic minimum information principle, using Extreme Physical Information (EPI) theory 
[14]. EPI theory is used in a simple and particular case to derive a second order system differential equation. This 
equation of motion reveals the phenomenon dynamics and its solution is a probability density function (PDF) with a 
power law tail. As an extension to the model a dissipative term is proposed in order to take into account that real 
I ■ systems are usually open, exchanging matter, energy, or information with their surroundings. 
• • ' Due to the fact that the proposed model is theoretic, it can be used to infer the possible PDF governing a system 
when its microscopic characteristics are known. The model can also be used to fit an observed distribution numerically. 
As will be shown later when fitting the tail of an observed distribution, the shape of the head is determined via an 
j_i , equation that relates the scaling parameter in the tail to the shape of the head. 

This paper is organized as follows: first, the Fisher information measure is presented; by applying EPI theory, then 
an information functional is constructed, solved, and the PDF of the system computed; next, a dissipative term is 
introduced into the information model; finally, the model is used to fit the head and the tail of 4 different power law 
tails that have been observed and already studied in the literature. 
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II. THE INFORMATION MODEL 



The model is based on EPI theory. This theory was developed by Roy Frieden, building on the basic information 
measure of Fisher information [15, 16]. In EPI theory, this measure is used to construct an information functional 
that is minimized to get an equation describing the motion of the system. 

In the literature EPI theory has been used to deduce several physics equations; for example, it was used to derive 
the Schrodinger equation in quantum mechanics and Einstein's field equations of general relativity [17]. It was also 
used to derive classical statistical physics in thermodynamics [18]. The EPI theory has also been applied to biological, 
economic and other complex systems, with interesting results [19-24]. In this paper, EPI theory is used to derive an 
analytic PDF of a system with a power law tail. The deduced PDF is a piecewise function with two parameters, one 
for the distribution head and the other for the tail. The major contribution of this model is to show that the two 
parameters are dependent. 



A. Fisher information 



The information measure used in EPI theory is Fisher information. This measure was derived in 1922 by R.A. 
Fisher from estimation parameter theory [25, 26]. In estimation parameter theory, an estimator 0(x) of an unknown 
quantity is unbiased if 

(e(x) - e) = j dx [Q(x) - @}f(x\Q) = o, (i) 

where, f(x\Q) is the PDF of the observed variable x given a parameter 9. This equation yields to the Cramer-Rao 
inequality 

e 2 > j, (2) 

where, e 2 is the mean-square error and / the Fisher information. This relation means the error of a measurement 
is bounded by the Fisher information. Developing Eq. (1), Fisher information can be written as 



=I dx m 



df(x\@y 2 



|0) V ^0 



(3) 



When the variable x obeys the shift relation x = + y, it is possible to simplify Eq. (3). If g(y\<d) is the PDF of 
y given then 



f(x\Q) = f(y + e\Q) 

= g(y\Q). (4) 

Finally, if random variable y was independent of the size of 0, Eq. (4) becomes 

f(x\e)=g(y). (5) 
Using Eq. (5) in Eq. (3) the Fisher information simplifies to 



where / represents the information retrieved through the measurement of variable y. The goal of EPI will be the 
computation of g(y) by means of an information balance. 



B. Application of EPI 



EPI theory involves the construction of an information functional to be minimized. The functional must have at least 
two terms. The first term is always the Fisher information Eq. (6) and the second term is the bounded information. 
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The Fisher information is the amount of information retrieved when parameter O is estimated measuring y; the 
bounded information is a term that can be constructed via a unitary transformation of the Fisher information. Here, 
the unitary transformation will be the conservation of the probability between the PDF g(y) and a second PDF h(z), 
given that y is a function of z. 

The model developed in this paper presupposes the existence of two observable variables, y and z, that are related 
by a the linear relation 



z = t{y)y, (7) 



where, t{y) is the piecewise function 



yyj \ t 2 , y > y y ' 

and where, t\ ^ and t 2 ^ are constants. 

If Eqs. (7) and (8) were known, then measuring y or z would produce a minimum information discrepancy. It turns 
out that the information functional T to minimize is the Fisher information I^ y \ the information retrieved when y is 
measured, minus the Fisher information I^ z \ when z is measured: 

T = J(») - k/W. (9) 

The constant k is a variable that measures the information discrepancy between measuring y or z in the system. 
Using Eq. (6) to write each term of Eq. (9), the information functional becomes 

r f , v'(y) 2 f , h '( z f , , 

^J dy wr- K J' h wf' (10) 

To minimize the functional J 7 , the second term on the right of Eq. (10) is written in terms of g'(y), g(y) and y: 

as shown in appendix A. 

Using Eq. (11) the information functional becomes 

T = J dyC[g'(y),g(y),y], (12) 

where 

is the Lagrangian of the system. The solution for g{y) is deduced applying Euler-Lagrange equations. To simplify 
further calculation, a change of variable is used, 

g(y) - q(y) 2 - (14) 

So, the Lagrangian becomes 

CW(yU{y),y] = W(y) 2 tt^^A ( 15 ) 

with solution 
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VM + ^ = «. (16) 

Eq. (16) represents the equation of motion of the system, and represents, therefore, the system behavior. 
The general solution of Eq. (16) is piecewise because function t(y) is also piecewise, so: 

Ky) \Q2(v), y>yo v ' 

The general form for solutions qi (y) and q2 (y) is 

Qi(v) = Ci,iy? +ki + cj,2y*~ fci , (18) 

where c^i and Cj t 2 are integration constants and coefficient fcj obeys 



i 

2V *?■ 



(19) 



The value of ij depends on the range of y through Eq. (8). The equivalence with the solution of a free quantum 
particle in quantum mechanics is straightforward, as shown in appendix B. 

In section IV, Eq. (16) will be complemented with a dissipative term to describe information flow between system 
and environment. 



III. A PARTICULAR SOLUTION 



To apply the model it is necessary to establish several boundary conditions. These conditions are usually taken 
from the real system under study. As an example, suppose a piecewise function t(y) with one step, as in Eq. (8). 
Also, suppose that q(y = 1) = 0, q(y — > oo) — > and k\ G C. The restriction over fci makes it possible to write q(y) in 
terms of Sines and Cosines, that is, in terms of an oscillatory wave. Obeying the piecewise function t(y), the solution 
for q(y) is, 



q(y) = { Cl K S ! n(fcl log(y)) ' 1 " V - yo , (20) 
I fc2 , y > y 

where the power law in the tail may be identified. The PDF of the tail can be written in terms of the typical Pareto 
distribution with scaling parameter a ta ii as 

q2(y) 2 = \c 2 \ 2 y 1 - 2k * = \c 2 \ 2 y-^ i ' +1 \ (21) 

with 

k 2 = ataa + 2 . (22) 

To compute coefficients c\ and c 2 in Eq. (20), the continuity of q(y) and q'(y) at y = y is used. These conditions 
make it possible to calculate the two coefficients in terms of y^, ki and k 2 . 
Taking into account that the PDF must be normalized, 

J q(y) 2 dy = 1, (23) 

an additional equation is deduced 

h = -fc 2 tan(fc 1 log(y )) . (24) 
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Figure 1: (a) Plots of the shape of Eq. (24); k\ values are the zeros of this function, (b) First 3 eigenfunctions of 
g(y): the ground state, dashed line; the second, dot-dashed line and the third, dotted line, (c) The complementary 
CDF in a linear plane and (d) in a Log-Log plane. The vertical line represents the yo value. 



Eq. (24) establishes a relation between the scaling parameter of the power law tail and the shape of the head. 
Using the normalization condition a relation between k\ and hi may be derived as a function of j/o- The relation 
will be particular to each system, often a transcendental equation with multiple solutions for k\. given the value of 
fc 2 and y . In such cases, when the equation gives several solutions for fci, solutions can be seen as eigenstates of 
the system. In few words, given a scaling parameter of the tail and the minimum value y n . it is possible to establish 
several eigenvalues for the shape of the head: the tail of the PDF fixes the head. Up to now, this relation has not 
been identified or studied analytically. 

Figure (1) shows Eq. (24), the PDF g(y) = q(y) 2 and the complementary cumulative distribution function (CDF). 
Function g(y) is plotted in its 3 first eigenstates. 



IV. DISSIPATIVE TERM 



Real systems, particularly social systems, are often conceived as open which means they exchange matter, energy 
or information with their surroundings. In this sense, a dissipative term H(q'(y),y) is proposed: 

W(y),?y) = /3— • (25) 

V 

Coefficient f3 will be the strength of the dissipation. This term is constructed by analogy with the Schrodinger equation 
in quantum mechanics, as shown in appendix B. 

Using this dissipative term, the equation of motion becomes 



(26) 
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The solutions q\ (y) and qi (y) now have the general form 



Qi(y) = Ci,iv 



,\-ki 



(27) 



where 



Ci t \ and Cj ; 2 are integration constants, and coefficient fc; obeys 




(28) 



with 




(29) 



q(y) 



d y x sin(fci log(y)), 1 < y < y 

C2y x ~ k \ y>yo 



(30) 



The scaling parameter a ta ii of the Pareto distribution can now be written in terms of the system characteristics 
and a dissipative term 



Using Eqs. (29) and (31) it can be inferred the information exchange direction with the surroundings: if A < 1/2 
the system gain information, if A > 1/2 the system loose information and if A = 1/2 the system is in equilibrium with 
the environment. In natural systems a direct equivalence with entropy changes can be made using Fisher information 
as an entropy measure [17]. 



This model can be used in a bottom-up approach, because the macroscopic parameters of an observed distribution, 
such as the scaling parameter and the y m i n value, can be related or explained using the microscopic parameters t\, 
t2, k and j/o using Eq. (19). This model can also be seen in a top-down approach, because it can be used to fit an 
observed PDF by means of numerical estimation of parameters. 

To fit an observed PDF, the scaling parameter a ta u and the y m i n value can be estimated using the maximum likeli- 
hood (ML) method as standard [27]. Then, using the corresponding relation equation derived from the normalization 
condition e.g. Eqs. (22) and (23), several eigenvalues k\ can be computed to describe the head. Because there could 
be several different values for k\, the head can be fitted using a linear combination of eigenvalues. 

In both approaches the model allows the interpretation of the parameters in terms of physical quantities or properties 
of the system. In this sense the model is not only descriptive but might become useful to control the system. By 
employing equations (28) and (31) the scaling parameter of the tail can be adjusted, and the shape of the head can 
be restricted to some particular values. 

Finally, it is apparent that this model was derived using a piecewise function t{y) Eq. (8), with only two pieces; 
but the solution can be generalized to n pieces. In this case the solution will be a piecewise solution with n different 
qi(y). A PDF with several scale parameters in the tail can be useful if the observed PDF shows more than one slope 
in the upper-tail. 



The model was used to fit 4 data sets with power law tails. The power law tails of selected data set were studied 
in [1, 27] (downloaded from http://tuvalu.santafe.edu/~aaronc/powerlaws/data.htm) and in [12], where world wealth 
distribution is simulated using an agent based model (ABM). 

To fit the whole PDF, first the scaling parameter a ta u and the y m in were estimated using an ML method, then 
the strength of the dissipative term and the shape of the head (a linear combination of the first two eigenstates) were 
estimated by direct minimization of the Kolmogorov-Smirnov statistic. 

The data set comprised: 



atau = 2 (k 2 - A) - 1. 



(31) 



V. 



APPLICATION OF THE MODEL 



A. Fitting the real data set 
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Data set 


t(y) 


ymin 


Clta.il 


A 


Information flow 


fci 


Dkl 


D C 


1 


One step 


228000 


1.2(1) 


-0.1 


In 


(0.23, 0.46) 


0.04 


0.11 


2 


Two steps 


50580 


1.36(5) 


0.2 


In 


0.41 


0.0099 


0.011 


3 


Two steps 


321 


0.77(2) 


2.0 


Out 


0.72 


0.0140 


0.0144 


4 


One step 


614 


1.68(5) 


0.25 


In 


(0.44, 0.89) 


0.007 


0.023 



Table I: Boundaries conditions and estimated parameters for each data set. The one step function is a 2 piecewise 

function as in Eq. (8); the two steps functions are a 3 piecewise function. The presence of two values for the k\ 
parameter means that the fit of the head was made with the two first eigenstates. Dkl is the Kolmogorov-Smirnov 
statistic measured as the maximum difference between the observed CDF and the analytical CDF. Dc = 1.627 1 \fn 
is the Kolmogorov-Smirnov goodness- of-fit test critical statistic, from which the model is rejected, so it is necessary 

that Dkl < Dc (where n is the data length). 

1. The numbers of customers affected by electrical blackouts in the United States between 1984 and 2002 (211 
registers). 

2. The human population of US cities in the 2000 US Census (19447 registers). 

3. Peak gamma-ray intensity of solar flares between 1980 and 1989 (12773 registers). 

4. World wealth distribution simulated using an ABM (5000 agents). 

Table (I) shows boundary conditions and estimated parameters. Figure (2) shows fitted distributions. Plots were 
made in a linear plane to observe the fit of the head and in a Log-Log plane to observed the fit of the tail. 

VI. DISCUSSION 

In the study of data set probability distribution functions, two basic filters are usually applied. One of these involves 
discarding the extreme and rare events. These events are considered to be outliers because they do not seem to follow 
the general tendency. Indeed, using Gaussian statistics it can be seen that extreme events make the variance too wide 
and the results less significant [28, 29]. As a result, the tail is excluded from the analysis. Another frequently used 
filter discards the events of the head, because the main issue is the power law tail study. The study of power law tails 
is an interesting research area because properties such as self-organized criticality and scale-free can be ascribed to 
the system [30-32] . In this kind of study, frequent events are excluded by means of a threshold value that renders the 
power law tail plausible. 

In this paper a model is presented that describes the whole distribution, involving the head and the distribution 
tail, and no data is discarded. Moreover, a relation between the scaling parameter of the power law tail and the shape 
of the general tendency of the head is also derived. 

The model constructed becomes interesting for the study of many systems that exhibit power law tails, because it 
proposes an equation of motion that describes the behavior of the system and allows its parameters to be interpreted. 
Even if the model has a piecewise restriction function Eq. (8) it can be used to explain and fit several systems, in 
particular social systems. A lot of social variables, such as prices, taxes, incentives, budgets, etc., follow piecewise 
restrictions. The model can also be used to gain insight concerning microscopic processes in natural systems that 
follow power law tails and are not already well understood 

The analysis and fitting of power law tails is an active research area. The computation of the scaling parameter 
of the PDF and the 

y-min value (value from where the power law is plausible) is not as trivial as it seems [27]. The 
model developed here contributes to this research area: first, by taking into account all the registers in the data set; 
second, by offering a new interpretation of the scaling parameter and third, by providing a relation between the tail 
and the head. 

A dissipative term was introduced into the model. This term permits the description of a large number of phenom- 
ena, and it should be interpreted in terms of the particular scenario of the system under study. For now, a simple 
interpretation is given in terms of the flow direction of the dissipation. A gain of information can be interpreted, as 
order is been established on the system, i.e. data sets 1, 2 and 4; a loss of information can be interpreted as the 
system seeking its thermodynamic equilibrium state, i.e. data set 3. 
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(a) The numbers of customers affected by electrical blackouts in the United States between 1984 and 2002. 
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(b) The human population of US cities in the 2000 US Census. 
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(c) Peak gamma-ray intensity of solar flares between 1980 and 1989. 
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(d) World wealth distribution simulated using an ABM. 
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Figure 2: Dots are the complementary CDF for each data set. The line is the fitted model. On the left, a linear plot, 
on the right, a Log-Log plot. The vertical line represents the y m i n value. 
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VII. CONCLUSIONS 



Using EPI theory according to the basic information principle, the equation of motion and the system PDF with 
a power law tail are deduced. The calculated PDF is as useful in describing the tail as the head of the PDF. In 
particular, the head is described as a combination of several eigenstates. This model, in a bottom-up approach, 
gives an explanation of the macroscopic parameters, a ta ii and y m i n , and the shape of the head, from the microscopic 
parameters of the system. In a top-down approach the model is equally useful in fitting the tail as the head of an 
observed PDF, giving an interpretation for the estimated parameters. 

The principal contribution of this paper to the analysis of systems with power law tail behavior concerns the relation 
between rare events in the tail and frequent events in head. In the literature, no one denies that there must be a 
relation between the head and the tail; here, a relation is proposed, by means of eigenstates. With this model it is 
not necessary to filter the data set and discard events: it is possible to study the general behavior in the head and 
take into account that the system presents complex characteristics as a power law in the tail. 

Not only will the proposed dissipative term give a better fit in a top-down approach, but it permits a description 
to be made of the interactions between the systems and their surroundings. Using this model the scaling parameter 
in the tail could be understood as a combination of the structure and properties of a system and its interaction with 
the environment. 

We believe that our model is capable of taking into account the fact that in many systems behavior is a mixture of 
order and disorder. It is also possible to gain insights regarding the plausible causes of emergent behavior, given the 
way parameters are fed into the model. 



Appendix A: Bounded Information 

To write the information functional 



in terms of g'(y), g(y) and y, the term 
must be re-written. 

The conservation of the probability is applied, using Eq. (7) and (8). Supposing that y and t(y) are random 
variables the PDF h(z) can be written as the product of two random variables; by definition 



h{z = t{y)y) = f dy^-g(y)p(-\y), (A3) 

J \y\ v 

where the function p(t\y) is the probability of t = t\ or t = t 2 , given y. This probability function, p{t\y), can be 
written in term of the delta function S(y — yo) and the Heaviside function H(y — y n ) as 



p(t\y) = ai S(t-h){l- H{y-y ) 

+a 2 S(t-t 2 )H(y-y ), (A4) 



where a\ + a 2 = I. 

Using equations Eq. (A4) in Eq. (A3) we get 



h(z) = J dy ^g(y) |a,i<5(J - *i)(l - H(y - y ) + a 2 5{^- - t 2 )H{y - y ) 
o 

Do oo 

/l z f 1 z 

dy y-g(y)a 1 S{- - h) + / dy —g(y)a 2 6(- - t 2 ). (A5) 
I y i y j \y\ y 



yo 
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Now h'(y) is calculated taken the derivative £^ of Eq. (A5), 

yO oo 

/ dy W\ giy)ai ~ h) + l dy ¥\ giy)a2 ^ 6{Z y h) 

yo 

yO oo 

[dy^-g(y)a 1 ^S(z-t 1 y)- f dy ±-g(y)a 2 ^6(z - t 2 y) 
J \y\ z J \y\ z 



yO 

dhjz) 
dz 

o 



--h{z). (A6) 



This last development takes into account that g(f:) = Uh(z). 

r j~ _ 

h(z) 



Finally, using Eqs. (A6) and (7), we write 1^ = J dz as 



I {z) = / dz 



Hz) 



= /< 

1 

yo oo 

f , , Hhy) , f , . h(t 2 y) 

= J dytl l^F + J dyt2 lW 



= l dy W^ 

where z* = tiy and z * * = t 2 y a , and h(Uy) = j-g(y). 

Appendix B: Dissipative term 

To introduce a dissipative term into the analysis it is only necessary to equate the second order differential equation 
Eq. (16) to a function H(q'(y),y), where H(q'(y),y) has a general form of a dissipative term, that is, in terms of 
q'(y). Here, a particular form for the dissipative term is inspired by the mathematical equivalence of Eq (16) with 
the Schrodinger equation. Under the change of variable y = log y, the Schrodinger equation of a free particle 

<up'\y) + hp(y) = 0, (Bl) 

becomes 

a q ^(y) + a q -M + b ^ = . (B2 ) 

y y 2 

In the same way, the general solution of the Schrodinger equation 

= Acxp(ky) + Bcxp(-ky) , (B3) 

becomes 

q{y)=Ay k + By- k . (B4) 

In both cases, k = ±y / —b/a. 
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Comparing Eq. (B2) with Eq. (16) an additional term a^-^- can be identified. Because this term has a first deriva- 
tive, by analogy with dissipative forces we propose that o,^-^- can be related to a dissipative source of information. 
So, by analogy with a quantum system we propose 

H(q>(y),y) = (3^-, (B5) 

where, (3 is a constant, related to the strength of the dissipation. 

The dissipative term is still not well understood, but allows a sharp estimate to be made of the scaling parameter 
at the tail: see Eq. (30). It is also interesting to discuss a new interpretation for the scaling parameter of the power 
law tail using Eq. (31). 
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